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lasting (post-supervision period) effect than regular supervision; 
that measurement scales can be developed for predicting which 
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the continuing effectiveness of intensive supervision, it may be 
essentisa to provide counselors with periodic morale-boosting 
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Final BvaXuatior Rapoct 
BMcutlva Sumnaxy 

For three and one-half years, North Carolina haa conducted a randomized 
experimental program designed to provide intensive supervision services for 
undisciplined youths (status offenders) placed under the protective 
supervision of the juvenile courts. Updated results from an ongoing 
systcvnatic evaluation of the project indicate that, for undisciplined youths 
with no prior history of court referrals for delinquent acts, the Intensive 
Protective Supervision Program provides a reduction of IS to 20 percent in the 
rate of progression to delinquent offenses during the supervision period and a 
cumulative {supervision period plus up to one year after supervision) 
reduction of 25 to 30 percent — relative to the corresponding rates observed 

* 

for youths in regular protective supervision. Experimental results also 
indicate: that early intervention {i.e., for status offenders who have not yet 
been referred to the courts for delinquent acts) facilitates reduction in 
subsequent delinquent behavior; that the reduction occurs primarily in 
referrals for nonfelony delinquent offenses; that there also are intensive 
supervision effects on reducing the runaway and truancy offense recidivism 
rates; that intensive supervision has a longer lasting (post-supervision 
period) effect than regular supervision; that measurement scales can be 
developed for predicting which characteristics of a youth client and her/his 
family are likely to lead to success in the intensive supervision program; and 
that, for the continuing effectiveness of intensive supervision, it may be 
essential to provide counselors with periodic morale-boosting attention and 
training in counseling and therapeutic methods to guard against the 
possibility of counselor burnout . 
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COORT COtmSBLOVS ZNTENSIVB SUPERVISION BXPERZMENT, PBASB XXZ: 

Final Evaluation Report 

Xntrodnetion 

For the past three and one-*hal£ years^ Ks^rth Carolina has conducted a 
randomized experimental program designed to evaluates the merits of providing 
Intensive supervision services for undisciplined yowt^ff or Offggndesa 
placed under the protective supervision of the juvenile courts. By 
definition, an undisciplined youth is a juvenile less than 16 years of age who 
(a) has run away from home (l*e*r is a runaway), (b) Is unlawfully absent from 
school (I.e., is a truant), (c) is re<|uL^rly disobedlant to his parent, 
guardian, or custodian and beyond their disciplinary control (I.e., is 
ungovernable) , or (d) is regularly found in places where it is unlawful for a 
juvenile to be. The essential idea of the Intensive Protective Supervision 
Project (IPSP) is that through the intensive supervision (by court counselors) 
and provision of professional services to status offenders it may be possible 
to decrease the rate of occurrence of additional status offenses and the 
likelihood that the youths will commit more serious delinquent offenses* 
Thus, as compared to regular protective supervision of status offenders by 
court counselc the IPSP involves more extensive and proactive contact 
between the couuse r, the status offender, and the status offender's family. 

Since November, 1987, an experimental project has been conducted to assess 
whether or not Intensive Protective Supervision (IPS) achieves the goals of 
reducing undisciplined acts and progression to delinquent behavior relative to 
Regular Protective Supervision <RPS) . This research has been supported by the 
state of North Carolina, Administrative Office of the Courts, Juvenile 
Services Division, Four juvenile court sites have participated in the 



experimental project aince November 1, 1987: Di9trict 10 <Sfake 
County/Relel9h) , District 18 (Guilford County/Greenaboro) , District 19A 
(Cabarrus and Rowan Counties /Concord and Salisbury)^ and District 28 (Buncombe 
County /Asheville) . Beginning November 1, 1989, the experiiront and its 
evaluation were expanded to £our new sites from which data on small numbers of 
cases now are available: District 5 (New Hanover County/Wilmington) , District 
ISA (Alamance County/Graham) , District 23 (Wilkes County/wilkesboro) 0 and 
District 27A (Gaston County /Gas tenia) . 

At each siter any youth adjudicated undisciplined after the starting date 
of the experiment who received a protective supervision disposition and who 
was not already under the court's supervision for a delinquent or 
undisciplined offense was put into a pool from which experimental (IPS) and 
control (RPS) group assignments were made. A key aspect of the experiment is 
that these assignments were made according to a well-Klef ined randomized 
procedure. This random assignment of juvenile clients ensures that the 
experimental (treatment) and control (comparison) groups are approximately 
equivalent with respect to such extraneous client characteristics as age, sex# 
race, and personality characteristics. This, in turn, makes the statistical 
comparison of ou :>mes from the two groups more reliable and accurate. 
Randomization also makes more plausible the attribution of group differences 
in outcomes to the experimental treatment procedure. 

Objective of the Report 

Previously, the present project evaluation team has provided statistical 
intact assessments of the Intensive Protective Supervision Project after 
approximately 1*3 years of operation ( Phase 1 Fin^il Evaluation Rgpogt/ dotted 
June 1, 1989) and again after approximately 2.5 years of operation (filiaaft II 



Final Evaluation Report, dated June 1, 1990) as well as at various intervening 
dates in Int3rim Evaluation Reports* Results £rom the Phase I ixqpact and 
process evaluation also were published separately in a peer-reviewed 
evaluation research journal (Land^ McCall, and Williams, 1990) . 

The purpose of the present EUaSft III EiMi SVfl^W^tj,Qn fiSBfi£L i3 to provide 
an updated statistical analysis o£ data from the Project as of May 15, 1991. 
Our updated tables now include results on all juvenile subjects processed 
through the Intensive (treati^.^nt, experimental) and Regular (control) 
Protective Supervision groups through mid-May 1991 — which encoicfiasses 
roughly 3.5 years of operation of the Project at the four original 
experimental sites and approximately 1.5 year of operation at the four new 
sites. The present doctiroent describes these tables and compares thwi with the 
tables in the June 1990 Phase II Final Evaluation Report and with similar 
tables in our March 1991 Phase III Interim Evaluation Report. In addition, 
a copy of a draft chapter based on our extended evaluation of the IPSP — 
which is to be published separately is attached as Appendix A (Land^ 
McCall, and Williams, 1991). 

Statiatlcal Description of the Clients 

Table 1 contains relative frequency distributions (percentages) and raw 
frequency distributions (cell counts in parentheses) on all 464 experimental 
and control group cases for which we have data as of mid-May 1991. Note that 
all of these 464 client cases have been admitted into the Intensive Protective 
Supervision Experiment from November 1987 to this date, but, of these, only 
360 have been closed . Closed cases are cases that have successfully coii¥>leted 
the assigned supervision {usually for a six months-to-a--year period), aged-out 
of the program, moved out of the district, refused (or their parents refused) 



Table 1. Horth Carolina Court Counaelor'a Intonsivo Proteetivo Sopervlaion 
BxperlBont: Praqooncy Dlstriliatlon on Selaeted Charaetariatlos 







Exneriaental 


Control 


Tot.ai 


Total Sanole 


52.4* 


(243) 


47.6* 


(221) 


100.0«(464) 


District 












9, Vilnlngton 


4.9^ 


' (12) 


7.7* 


' (17) 


^.3* (29) 


10, Raleigh 


15.6 


(38) 


18.6 


(41) 


17.0 


(79) 


18A» Grabaa 


5.8 


(14) 


7.2 


(16) 


6.4 


(30) 


18, Qreenaboro 


20.6 


(50) 


20.9 


(46) 


20.7 


(96) 


19A, Concord/Salisbury 


16.9 


(41) 


12.2 


(27) 


14.7 


(68) 


23. Wllkaboro 


9.r 


(24) 


9.0 


(20) 


9.5 


(44) 


27A, Gastonla 




(19) 


8*6 


(19) 


8.2 


(38) 


28, Aabeville 




(45) 


15.8 


(35) 


17.2 


(80) 


Afe 

7.8,10-12 


13 . 1 


(32) 


8.1 


(18) 


10.7 


( 50) 


18 


16.0 


(39) 


19.9 


(44) 


18.0 


( 83) 


14 


38.8 


(94) 


39.4 


(87) 


39.0 


(181) 


lS-16 


22.1 


(78) 


32.6 


(72) 


32.3 


(150) 


Sex 














Peaale 


68.3 


(166) 


69.2 


(153) 


68.8 


(319) 


irale 


31.7 


( 77) 


30.8 


( 68) 


31.2 


(145) 


Race 














White 


AO IK 


(169) 


70. 1 




69.8 


(324) 


Black 


28.9 


( 70) 


29.4 


( 65) 


29.1 


(135) 


utner 


1.6 


( 4) 


0.5 


( 1) 


1.1 


( 5) 


Reason Referred 














Runaway 


52.7 


(128) 


52.9 


(117) 


52.8 


(245) 


Truancy 


31.7 


( 77) 


30.8 


( 68) 


31.3 


(145) 


Ungovernable 


15.6 


( 38) 


16.3 


( 36) 


15.9 


( 74) 


Nuaber of Prior 














Status Referrals 














0 


80.3 


(195) 


85.5 


(189) 


82.7 


(384) 


1 


15.2 


( 37) 


U.3 


( 25) 


13.4 


( 62) 


2 


3.3 


( 8) 


2.3 


( 5) 


2.8 


( 13) 


3 


1.2 


( 3) 


0.9 


( 2) 


1.1 


( 5) 


Nuaber of Prior 














Dellnouent Re^efrals 














0 


90.2 


(219) 


93.2 


(206) 


91.5 


(425) 


1 


7.4 


( 18) 


5.9 


( 13) 


6.7 


( 31) 


2 


1.6 


( 4) 


0.0 


( 0) 


0.9 


( 4) 


3 


0.8 


( 2) 


0.9 


( 2) 


0.9 


( 4) 



^As a percentage of total cases (464). 
As a percentage of total experlaental cases (243). 
^As a percentage of total control cases (221). 
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to continue participation, or been adjudicatedl delinquent on a aub^equent 
petition to the courts For purposes of comparlaonr Table 2 reports the aame 
frequency distributions for the closed cases as Table I does for all admitted 
cases. Much of the analysis below will be restricted to the closed cases. 

Frequency distributions on the following client characteristics are given 
in Table 1 and Table 2: experimental versus control group assigmoentf 
districtr age, sex, race, reason referred (to the juvenile court) , nuiober of 
prior status offense referrals, and number of prior delincjuent referrals. The 
first rows exhibit the percentage and numerical allocations of clients to the 
experimental and control groups. The right-hand columns of the tables 
contain the relative frequencies a r percentages of the total client sample as 
of mid-May 1991. The other two columns then give the corresponding relative 
frequencies as percentages of the total experimental and control cases « This 
facilitates conparisons of the frequency distributions within the latter two 
groups with those of the total sample. 

As was the case in the June 1990 Phase II Final Evaluation Report, the 
distribution of admitted cases in Table 1 between the experimental and control groups 
remains roughly equal {52.4 percent in the experimental group and 47.6 percent 
in the control group) . The distribution of cases among the four original 
sites also continues to remain relatively stable — with District 18 
(Greensboro) having the most {96) and District 19A {Concord/Salisbury) the 
least (68) . Among the four new sites, Wilkesboro and v^astonia have the 
largest number of cases assigned at this point (with 44 and 38 cases, 
respectively) followed by Graham (30) and Wilmington (29) . 

With respect to the frequency distributions of personal characteristics of 
the juvenile clients iage, sex, race), it can be seen from Table 1 that the 
percentage differences between experimental and control groups usually are 

ERIC ^ 8 



small — on the order of 3 percentage points or less. In general, various 
differences between the groups in the percent distributions on these 
characteristics that we have noted in previous reports have declined somewhat. 
This is consistent with a stabilization of the frequency distributions as the 
sample sizes increase under the operation of a proper random assignment 
procedure. 

This narrowing of percentage differences between the two groups also 
extends to the courts-related characteristics* In particular, the relative 
frequencies of the groups by reasons referred (to the juvenile court) are 
quite comparable* Similarly^ the difference in percentages of the two groups 
who have had one or more prior referrals to the juvenile court for status 
offenses has narrowed from as high as 8 percent in previous reports to about 5 
percent in Table 1, Finally, the difference in the percentages of clients 
having one or more prior delinquent court referrals has declined from as high 
as 6 percent in previous reports to 3 percent in Table 1. 

Despite this general narrowing of differences between the groups, however, 
the percentage of clients with SLL m£& &£i££ delinquent ta,fft^gal8 remains 
somewhat higher in the experimental than the control group {9.8 versus 6.8 
percent, respectively) . Nuwerically, there are 24 such cases in the 
experimental group versus 15 in the control group — which means that the 
experimental group has 60 percent more cases (with one or more prior 
delinquent referrals) than does the control group. Even more significantly, 
the experimental group contains 6 cases with two or more prior delinquent 
referrals — those with very high probabilities of additional delinquent 
offending — compared to only 2 cases for the control group. 

The foregoing comparisons of differences in the frequency distributions of 
cases by experimental versus control groups suggest that the two groups have 



become niore similar during the laat year and a half with respect both to 
personal aud court-related characteristics. Indeed, nith the exception of the 
experimental group continuing to have somewhat higher percentages of ""tough** 
cases (i.e«, cases with higher-order prior offense histories that are more 
likely to recidivate with status and/or delinquent offenses), the groups are 
quite conqparable. These differences are not large and could well be due to 
chance variation associated with the random assignment of cases to the 
experiiMintal and control groups. They do suggest, however, that it is very 
important to remain sensitive in the outcome evaluation to these sources of 
heterogeneity in the experimental and control groups and to continue to 
analyze separately those groups that are as homogeneous as possible with 
respect to prior offense histories. 

The frequency distributions for all cloaed ssssa^ reported in Table 2 are 
closely similar to those for all admitted cases in Table 1. Most of the 
percentages by specific characteristic differ by less than 2.0 between the two 
tables. The msi&l filfiSlfid £§aa in both the experimental and control groups is 
aged 14 to 16 (over 70 percent of cases), female {slightly over two-thirds of 
cases), white {also just over two-thirds of cases), and referred to the 
juvenile court for runaway or truant behavior {about SO and 30 percent of 
cases, respectively) . The fact that these modal characteristics and the 
associated frequency distributions for closed cases in Table 2 are very 
similar to those for admitted cases in Table 1 attests to a lack of selection 
bias due to participation refusals, moving out of the district, and so forth* 
Again, however, it merits noting that the number of closed cases in the 
experimental group with -^ne or more prior delinquent court referrals is 23 as 
compared to only 13 for the control group — and 6 closed experirontal group 

^ 6 

ERIC iU 



TBbl« 2. Itorth Cwollna Court GooBMlor's IntttasJIw Prot«etl«« Sopanrislon 

BiperiMBt: Wtmsaawf OlstriboUoa on SeleeM Cterceterlstlcs 
for all Cleaad Cmo m of Iter 19ftl. 



Total SaaQle 


.a 
83.1 


' (191) 


-.a 
46.9 


' (169) 


100.0«(360) 


District 


- «b 




* «b 






8. Nllalngton 


4.2 


( 8) 


8.9 


' (15) 


6.4* (23) 


10, Ralel^ 


17.3 


(33) 


21.9 


(37) 


19.4 (70) 


19A, Qrabaa 


5.8 


(11) 


4.1 


( 7) 


5.0 (18) 


18. Grwnsboro 


23.5 


(45) 


21.3 


(36) 


22.5 (81) 


19A, Concord/Salisbury 


16.2 


(31) 


13.0 


(22) 


14.8 (53) 


23, Wllksboro 


8.4 


(16) 


7.7 


(13) 


8.1 (29) 


27A, Gas tenia 


6.3 


(12) 


7.7 


(13) 


6.9 (25) 


28, Ashevllle 


18.3 


(35) 


15.4 


(26) 


16.9 (61) 


Age 

7.8.10-12 


12.4 


(24) 


6.5 


(11) 


9.8 ( 35) 


13 


14.8 


(28) 


17.8 


(30) 


16.1 ( 58) 


14 


37.8 


(72) 


42.6 


(72) 


40.0 (144) 


15-16 


35.0 


(67) 


33.1 


(56) 


34.1 (123) 


Sex 












Peaale 


67.0 


(128) 


68.6 


(116) 


67.8 (244) 


Male 


33.0 


( 63) 


31.4 


( 53) 


32.2 (116) 


Race 












White 


68.1 


(130) 


69.8 


(118) 


6B.9 (248) 


Black 


29.8 


( 57) 


30.2 


( 51) 


30.0 (108) 


Other 


2.1 


( 4) 


0.0 


( 0) 


1.1 ( 4) 


Reason Referred 












Rttnaitay 


52.9 


(101) 


55.0 


( 93) 


53.9 (194) 


Truancy 


31.4 


( 60) 


30.2 


( 51) 


30.8 (111) 


Ungovernable 


15.7 


( 30) 


14.8 


( 25) 


15.3 ( 5") 



Nuaber of Prior 
Status Referrals 

0 

1 

2 

3 



80.6 (154) 
13.6 ( 26) 
4.2 ( 8) 
1.6 ( 3) 



85.2 (144) 
10.6 ( 18) 
3.0 4 5) 
1.2 ( 2) 



82.8 (298) 
12.2 ( 44) 
3.8 ( 13) 
1.4 ( 5) 



jfuaber ££ Prior 
Deilnouent Referrals 

0 

1 

2 

3 



88.0 (168) 
8.9 ( 17) 
2.1 ( 4) 
l.O ( 2) 



92.3 (156) 
7.1 ( 12) 
0.0 ( 0) 
0.6 ( 1) 



90.0 (324) 
8.1 ( 29) 
1.1 ( 4) 
0.8 ( 3) 



a 



.As a percentage of total cases (360). 
^As a percentage of total experlaental cases (191) 
As a percentage of total control cases (169). 
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eaaes itith two or more prior delinquent referrals as compared to 1 such 
control group case. 



Outcona Svaluation: Dlffarence-of-tteans Analyses 

Supervlsion-*Period CMPparlsons . Table 3 is an updated version of the iraan 
outcome con^arisons of experimental and control groups contained in our 
March 1991 Interim Evaluation Reports The three outcome variables for %fhich 
mean rates <of cases closed as of May 15, 1991) are reported in Table 3 are: 
the delinquent offense r^t^ (DELOFF) coit¥>uted as the percentage of cases in 
the group that had one or more court referrals for delinquent acts dwr4nq the 
supervision period; atfltiia offense ^^ffj^t^V^sm (STATOFF) computed as the 
percentage of cases in the group that had one or more status offenses reported 
during the supervision period; and overall g.ygg^^a (SUCCESS) confuted as the 
percentage of cases in the group for which the counselor's judgment is that 
overall the client successfully con^leted the supervision period. 

Table 3 is arranged in three panels ^ each of which refers to an 
identifiable population of relatively homogeneous clients. Panel A reports 
the mean rates on the three outcome variables for both the experimental and 
control groups for those youth clients with prior court referrals for 
delinquent offenses, while Panel B displays the rates for youths with qsl P^ior 
delinquent court referrals. Because our previous reports have shown that 
Intensive Pro:ective Supervision is more effective relative to Regular 
Protective Supervision for youths with no prior delinquent referrals, Panel B 
is of principal interest. For comparison with subsequent tables. Panel C also 
is defined for youths with no prior delinquent offenses — but only for closed 
cases from the four original sites. 
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TwbH 3. Itfrth OnroliBa Omrt GovBMlor't lBt«nlv» ProtMstlvs taperrisioii 
Btperiaeiit: Meaci CF»re«iti«M) on tto«e OvtcoM VariablM for 
Fsriod of SopenpisiOB Only for Closed Cmoo — BiporlMital 
and Control OroBP« CBnorod> 



Otttcoae 
Variables 


Means in 
Bxperiaental 
Group 


Means in Differences 
Coi^IaoI of 
Group Means 


t-8tatlStiC8 
(one- tailed 
p values) 


Panel A. Youths with Prior Delinquent Offenses 


DBLOPP 

STATOPP 

SUCCESS 


S2 . 2 
39.1 

eo.9 


15.4 36.8 
23.1 16.0 

61.5 -0.6 


Saople 
sizes 
too saall 
for 

slpiificance 
tests. 


8 


23 


13 




Panel B. Youths with No Prior Delinquent Offenses 


DELOFF 


20.2 


24.4 -4.2 


-0.89 
(p < .20) 


STATOPP 


19.6 


22.4 -2.8 


-0.62 
(p > .25) 


SUCCESS 
N 


66.7 
168 


60.3 6.4 
156 


1.20 
{P > .15) 


Panel C. 


Youths with No Prior Delinquent Offenses — Four Original Sites Only 


DELOPP 


20.2 


27.0 -6.8 


-1.23 
(P < .15) 


STATOPP 


20.2 


27.9 -7.7 


-1.39 
(P < .10) 


SUCCESS 
N 


68.6 
124 


60.4 8.2 
111 


1.66 
(p < .05) 



Note: See text for definitions of the outcoae variables. 

i 
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Zt will be seen that Panel A (for youcns with one or more prior delinquent 
oCfensea) reports higher mean ratea on both the DELOFF and the STATOFP outconie 
variables for the experimental (Intensive Protective Supervision) than Cor the 
control (Regular Protective Supervision) group. This is consistent with the 
presence noted above of relatively more experimental than control group cases 
with two or more prior delinquent court referrals and the higher probability 
of delinquent recidivism of such cas^ss* Because of this and the relatively 
small numbers of cases in this panel, further statistical analysis of Panel A 
is not appropriate* 

More meaningful comparisons can be based on Panels B and C of Table 3 ~ 
which are restricted to those youths with no prior delinquent offenses* With 
respect to the delinquent referral rate (DELOFF), Panel B shows a mean 
difference of 4.2 percentage points in favor of the experimental (Intensive 
Protective Supervision) group. This compares to a mean difference of 6.8 
percentage points in favor of the experimental group reported in Panel C for 
the four original sites* On the status offense outcome variable (STATOFF)r 
Panel B displays a 2,8 percentage point difference in favor of the 
eitperimental group, while this difference is 7.7 percentage points in Panel C. 
For the third outcome variable, the counselor' s judgment of whether the 
supervision period was succassfuli overall (SUCCESS) # Panel B shows an 6.4 
percentage point difference in favor of the experimental group, while Panel C 
estimates this percentage point difference as 8.2 ♦ Several comments are in 
order regarding these percentage differences in the outcome variables between 
the two groups* 

f if some concern was expressed in previous Interim Evaluation Reports 
(especially the October 1989 diri February 1990 Phase II Reports) about the 
apparent deterioration in tne mean percentage differences in the DELOFF and 
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STATOFF outcome variables compared to our June X989 Phase I Final Evaluation 
Report (with cases closed as of May 1989} . In our June 1990 Phase II Final 
Evaluation and October 1990 Phase III Interim Evaluation Reports, we then 
nuted that this deterioration appeared to have ended. But, in our March 1991 
Phase III Interim Evaluation Report, the trend towards convergence appeared to 
have resumed. From Panel B o£ Table 3, it can be seen that this new trend 
towards convergence of the mean outcome variables has continued during the 
last three months. That is, the percentage point difference on the DELOFF 
outcome variable in Panel B of Table 3 (4.2) is less than the 5.0 points 
reported in March 1991. For the STATOFF outcome variable, the percentage 
difference in Panel B (2.8) is less than that observed in our October 1990 
Phase III Interim Evaluation Report (6.2) but slightly higher than the 1.9 
percentage point difference reported in March 1991. The latter might appear 
to be a reversal of the trend towards convergence, but both the DELOFF and 
STATOFF percentage differences reported in Panel C of Table 3 (6.8 and 7.7, 
respectively) are less than these percentages in March 1991 (7.6 and 6.5). 

In brief# for youths with no prior delincnaent offenses, the evidence 
suggests that there has been a renewed degradation of differences between the 
experimental and control groups on the delinquent and status offense outcome 
variables. How can this degradation be explained? It appears to be due to a 
combination of decreased delinquent and status offense rates for the control 
(RPS) group (from on t^i* order of 30 to 35 percent in October 1990 to on the 
order of 22 to 27 percent in Table 3) and, at le- 3t for the DELOFF outcome 
variable, an increased rate in the experimental (IPS) group (from about 14 
percent in June 1989 to about 20 percent in Table 3) . 

The decreased delinquent offending rate in the control group is consistent 
with fieid-^based observations and interviews reported in our June 1990 Phase 



II Pinal Evaluation Report — which suggested that some Regular Protective 
Supervision Counselors were attempting to use **experiii«ntal** s^thods in their 
work. The increased delinquent offending rate in the experimental group may 
be due to one or more of several factors r such as: 

— staffing changes among the IPS counselors at some sites; 

some decline in enthusiasm among these counselors as the novelty of the 
IPS process declined and the program became a routinized part of the juvenile 
court bureaucracy <a not unusual outcome in human services experiments) ; 

— some greater awareness of (and discounting about the seriousness of) 
the Intensive Protective Supervision treament procedures among juvenile status 
offending populations in experimental site areas; 

— and/or other systematic or random factors of which we are not aware. 
Additional analyses reported below will further probe this apparent 
degradation of the difference in the experimental and control group outcomes. 

gecond ^ none of the mean percentage differences in Panels B or C in Table 
3 reach conventional levels of statistical significance* On the other hand/ 
the 4.2 mean difference in the delinquent offense outcome variable in Panel B 
(youths with no prior delinquent offenses) does represent approximately an 
17.2 percent reduction in the rate of occurrence of this type of behavior as 
compared to that of the control group. This continues to be large enough to 
be of possible clinical (i.e., treatment) and policy significance. It is rare 
in juvenile justice to find programs that reach this order of magnitude of 
effects In addition^ there is some evidence that early intervention — before 
status offenders have been referred to the juvenile courts for delincfuent 
offenses — is more likely to have this effect in reducing delinquency. 

Third s it Should be observed that the overall tendency of experimental 
(IPS) Counselors to judge their youth clients as having successfully completed 
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the supervision period at higher rates than the control (EIPS) Counselors 
noted in our previous evaluation reports — - continues to be apparent in Panels 
B and C of Table 3* This tendency^ however^ has declined from on the order of 
a 20 to 25 percentage point difference in Phase I reports to on the order of a 
6 percentage point difference in Panel B of Table 3. Consistent with the 
remarks made above, this decline is due not primarily to a decrease in the 
rate at which the IPS Counselors judge their cases to be successful {for Panel 
B youths, this rate was about 71 percent in our June 1989 Phase I Final 
Evaluation Report versus 67 percent in Table 3) , but rather to an increase in 
the rate at which RPS Counselors judge their cases to be successful (from 49 
percent in the July 1989 Report to about 60 percent in Table 3) • 

Cumulatlva-Perlod Cooqparisons . The outcome variables for which group** 
specific means are reported in Table 3 are defined fiQlX £2£ thA ^rLod a£ 
flupervision (Intensive or Regular) — which typically is on the order of six 
months. A related question about the impact of the experimental treatment 
pertains to whether it has a continuing effect after the supervision period 
has ended (see comments by McCord, 1990, on the Land et al. 1990 publication 
of results from the Phase I Final Evaluation Report) . Analogously^ since the 
supervision-period effects have been reported in Table 3^ one can monitor the 
DELOFF and STATOFF outcome variables after supervision has ended and combine 
these post-supervision-period results with those of Table 3 to obtain 
cumulative outcome gfffftp^ri ^qpS- 

This is*, the type of comparison of outcomes reported in Table 4 — which 
gives the qumulative means on the outcome variables and mean differences for 
the same groups as in Table 3 £^ Ul£ combined period gypgyv^Sign Bnsi & 
post-suoervision follow-up period al M£ La fflOfi U&SiL S£XS£. tllS aaSSA &£& 
closed > Because only closed cases from the original four sites have been in 
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Table 4. Wwtth GarollM Court OomiMlor'* Intttslvs Pvotaetlvs Sqpervlaloii 
BiperlMat: Iteaiw (Psrooitavn) on Ihrve Ontcow Vurlablm tw 
Vom Mgloml Sites Only for Period of SnpervielOB and 0)^ to One 
Year After for Closed Cases — Bspertoental and Control Sroops 



Otttcoae 
Variables 



Means in Means in 
Experiaental Control 
Group Group 



Differences 
of 
Means 



t-8tatistica 
(one-tailed 
p vslues) 



Panel A. Youths with Prior Delinquent Offenses 



OELOPP 
STATOPP 

success 

N 



60.0 

45.0 

65.0 
20 



10.0 

30.0 

70.0 
10 



50.0 
15.0 
-5.0 



sixes 
too saall 
for 

siflpnificance 
tests . 



Panel B. Youths with No Prior Delinquent Offenses 



OELOPF 



STATOFP 



SUCCESS 



24.2 



25 • 8 



68.6 



124 



33.3 



34.2 



60.4 



111 



-0.1 



-8.4 



8.2 



-1.54 
(P < .10) 

-1.40 
(P < .10) 

1.66 
(P < .05) 



Note: See text for definitions of the outcose variables. 
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the escperintent sufficiently long to qualify for this type of con^airlaonr the 
cases compared in Table 4 are restricted to these four sites. 

It can be seen from Panel A that the only difference from Table 3 is in 
the slightly higher OELOFP rate for the experimental group (60 percent as 
conqpared to 52.2 percent in Table 3). This is consistent with the proposition 
that vrafcha with prior (^J^inquent offenses liMIX fiSSBOiL dftlilffiqwent 

offenses Sisxi^ SQSl thus. hMns. ihslJL SL&3S1 dsuloa th& 3wpegyia4gn 

On the other handr the mean difference on the DELOPF outcome variable in 
Panel B of Table 4 is more tht.n twice the corresponding difference in Table 3 
<9.1 versus 4.2 percentage points) r snd this mean difference is of marginal 
statistical significance. It has a probability-value (p-value) of less than 
0*10 on a one-tailed significance test. In other words^ on the presumption 
that the Intensive Protective Supervision treatment should reduce the 
delinquent referral rate in the experimental group as compared to the control 
group# a nman difference of the magnitude reported in Panel B of Table 4 vrould 
occur by chance less than ten times out of one hundred if the true mean 
difference were zero* 

These results imply that some of the youths in this panel progress to 
delinquent offenses during the post -supervision period and that those in the 
control group do so at higher rates than those in the experimental group. 
Thusr lJ2i£ experimental (Intensive) treatment ^ppgflrg ^ t^sSSL & mi& l^9tim 
im&Si. QSX tSiSi delinouent r^^g ih& S9niKQl iSSSlilASX. 

procedure . This conclusion is also supported by the fact that, during the 
post-supervision follow-up period, control group youths were involved in 
delinquent incidents at about twice the frequency of youths froir ihe 
experimental group (13 incidenc3 in the control group versus 7 in the 
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experimental group) • That is^ a given re£erral to the juvenile court may 
involve more than one delinquent incident r and in thia caae youtha in th^ 
control group referred for delinquent offended had been involved in nearly 
twice aa many auch reported incidents. 

For the STATOPF Outcome variable^ Panel B of Table 4 alao report a a mean 
percentage point difference of 6*4 in favor of the experimental group. Thia 
difference alao ia of marginal atatistical significance that is^ likely to 
have occurred by chance only 10 times out of 100 if the true difference 
between the groups were zero* Thus, experimental (Intenalv^) Prot^eefciva 

thsXk dfifi& th& control {Rgqw;i^y) &£2£fidU££* 

Offenae^Spaclfie Coiqpariaona . To further analyze the foregoing 
differences^ Table 5 reports additional supervision-period breakdowns of the 
OELOFF and STATOFF rateS/ while Table 6 displays the corresponding cumulative 
(supervision-period plus up to a ore-year follow-up period) resulta. In other 
wordSr Table 5 is an offense-specific analogue of Table 3 and Table 6 is the 
offense-specific analogue of Table 4 . In each case, the DELOFF rate is 
disaggregated into FELONY and NONFELONY (misdemeanor^ traff ic^ and city 
ordinance violations) categories and the STATOFF rate is deconqposed into 
RUNAWAY and TRUANCY categories • 

As was the case with the comparisons in Tables 3 and 4, the results in 
Tables 5 and 6 demonstrate modest differences in favor of the experimental 
treatment group for the supervision-period and^ for most comparisons^ larger^ 
more statistically significant differences in che cumulative-period 
con?>arisOuai« It also can be seen from these tables that fchQ p y;i,M ir Y 

cumulative impacts ^ ^ ^spgff^ment^; (l?S? ppgcedwiTgg seAativs iha, 

SSaSJUlX laeJSl procedures 4 lower {^ONfgLpNY oUSSkMS. SALai S&d XSMSSL RVWftWM 
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Table S. Hortb Carolina Court Counselor* a Intensive Proteotive Supervision 
Bxperlnent: Means 'Pereentages) on Felony » NlsdeBesnor. Rnnamiy, 
and Truanoy Oatcone Variablea for Period of Supervision Only for 

Closed Casea — Snerinental and Control Qronos Connared. 



Outeoae 
Variables 


Means in 
Experiaental 
Group 


Means in 
Control 
Group 


Differences 
of 
Means 


t-statistlcs 
(one-tailed 
p values) 


Panel A. Youths with Prior Delinquent Offenses 


FELONY 


21.7 


7,7 


14.0 




(iONFBLONY 
RUNAIfAY 


30.4 
30.4 


7.7 
15.4 


22.7 
15.0 


Saaple 

sizes 
too sjall 
for 

significance 
tests. 


TRUANCY 


17.4 


7.7 


9.7 




N " 


23 


13 






Panel B. Youths with No Prior Delinquent Offenses 


FELONY 


3.6 


3.9 


-0.3 


0.13 
(P > .25) 


NONPBLONY 


16.7 


20.5 


-3.8 


-0.89 
(P < .20) 


RUNAWAY 


14.9 


18.0 


-3.1 


-0.74 
(P < .25) 


TRUANCY 
N 


4.2 
168 


5.8 
156 


-1.6 


-0.66 
(P > .25) 


Panel C. 


Youths with No Prior Delinquent Offenses — Four Original Sites Only 


FELONY 


4.0 


5.4 


-?.4 


-0.49 
(P > .25) 


NONFELONY 


16.1 


21.6 


-5.5 


-1.07 
(p < .15) 


RUNAWAY 


15.3 


22.5 


-7.2 


-1.40 
(p < .10) 


TRUANCY 
N 


4.0 
124 


7.2 
111 


-3.2 


-1.05 
(p < .15) 



O Note: See text for definitions of the outcoee variables. 
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Ttble 6. North Carolina Court Counaolor'a Intenaiva Protaotiva Saparviaion 
Bxparlaant: Ifsana (Pereaatagat) on Paloay, Rmaiiay, uid Truaney 
Ootcoaa Variablea for Four Original Sitaa OnAy for Period of 
Soperviaion and 0p to One Year After for CZoaad Cataa 

Bnpariaantal and Control Qroaps Coapared. 



Outcoae 
Variables 


Ileana in 
Bxi^rlaental 
Group 


Means in 
Control 
Group 


Differences 
of 

Means 


t-8tatistic8 
(one-tailed 
p values) 


Panel A. Youths with Prior Delinquent Offenses 


FELONY 


35.0 


10.0 


25.0 




NONFEU)NY 
RUNAWAY 


25.0 
35.0 


0.0 
20.0 


25.0 
15.0 


Saaple 
sizes 
too saall 
for 

significance 
tests. 


TRUANCY 


25.0 


10.0 


15.0 




N 


20 


10 






Panel B. Youths with No Prior Delinquent Offenses 


FELONY 


e.5 


8.1 


-1.8 


-0.48 
(P > .25) 


NONFELONY 


17.7 


25.2 


-7.5 


-1.38 
(p < .10) 


RUNANAY 


19.4 


25.2 


-5.8 


-1.07 
(P < .15) 


TRUANCY 
N 


5.7 
124 


10.8 
111 


-5.1 


-1.43 
(P < .10) 


Note: See 


text for definitions 


of the outcoae 


variables. 





And TRUANCyy sSiS^Sa^* By comparison^ the cumulative FELONY rates show only a 
small difference. 

Before moving on to other analysesr it should be noted that the foregoing 
inferences based on Tables 3 to 6 presume that status and delinq[uent offenses 
either have been accurately and completely reported for youtha who 
participated in both the experimental and control groups or, if the records 
are incooqplete, they do not reflect a differential bias towards either group* 
That iSr if offense reports are missing, it must be assumed that they are 
missing at random between the two groups. We have no way of knowing with 
certainty whether or not this is a valid presumption^ and, indeed«r it probably 
is impossible to know in some absolute sense. On the other hand, efforts have 
been made by Administrative Office of the Courts personnel to ensure an 
accurate and conplete reporting of all offenses for youths in the IPS 
experiment, and field observations do not give any cause to suspect 
differential bias* Accordingly^ in the absence of information to the 
contrary, we believe that the inferences made above are not artifacts of 
differential reporting. 

Ootc«tte Evaluation: Regression Analyses 

Conplete-Sample Results, Regression analysis provides another way of 
examining the data with respect to possible effects of the experimental 
treatment on the outcome variables. The advantage of regression analysis is 
that it allows us to control simultaneously for the effects of several client 
characteristics as well as the experimental treatment. Thus, regression 
analysis yields estimates of the part;ial effects of the experimental treatment 
as opposed to the main ^^^ect^ estimated in the mean difference tables. 



Tables 7 and 8 report updated regression equations for the three outcome 
variables (DBLOPF, STATOPF, and SUCCESS) . Table 7 displays results defined on 
the supervision-period only, while Table 8 gives the coRq>arabIe cumulative 
{supervision period plus up to one-year follow-up) results » Mencer the 
regressions of Table 8 are estimated on closed cases from only the original 
four experimental sites. Each of the equations in these tables merits 
comment • 

As was the case in our June 1990 Phase II Pinal Evaluation Report^ the 
DELCi regression equations in Tables 7 and 8 continue to show statistically 
significant effects of the gender variable. That is^ female youth clients are 
less likely to be referred to the courts for delinquent offenses — during 
either the supervision or cumulative periods. As in the October 1990 Phase 
III Interim Evaluation Report^ the prior delinquent offenses explanatory 
variable also has a statistically significant coefficient in Tables 7. But 
the coefficient for this variable in Table 8 is smaller and not statistically 
significant. This difference is due to the fact, noted above^ that youths 
with prior delinquent offenses tend to be referred to the courts for new 
delinquent offenses earlier {i*e.^ while under protective supervision) than 
other youths. 

On the other hand, the cumulative DELOFF equation in Table 8 shows a 
significant negative coefficient for the age variable which indicates that 
youths assigned to protective supervision (either RPS or IPS) at an older age 
are less likely to be referred to the court for a delinquent offense than 
youths assigned at. a younger age. The age variable also has a significant 
coefficient for the supervision-period results in Table 7. It is plausible 
that these negative age effects are due to an "ageing out'* decline in 
offending behavior of older youtn clients. 
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Tabl« 7. Nortli Carolina Court Coaiiaelor*t Intenalvo Proteotlvo Suparvlalon 
BsperlMBt: Regreasion Analyaea of Throe OoteiMo vsarlablaa for 
Porlod of Soponrialoii for Cloaad Caaoa: Natrio Ragroaaioa 
Coefflolanta. t ratioa (In parantlwaoa), and Standardlsoil Segreaaion 

Coefflcianta in bracketal. 



Ontgone Variablea 

Independent 

Variables DELOFF STATOFF SUCCESS 



Group 

(axper . "1 » control <"0) 



Age (years) at last 
birthday at entry 
into protective 
aoperviaion 



-.008 
(0.180) 
(-.009] 

-.038 
(1.844) 
[-.097] 



«• 



-.014 
(0.315) 
1-.017] 

-.035 
(1.735) 
[-.092] 



O.OSO 
(0.742) 
[0.039] 

0.008 
(0.228) 
[0.O12] 



Sex 

(fenale-1 .aale-0) 



Race 

(black«*l »nonblack<*0) 



Prior Delinquent 

Offenaea 
( l-priors , O«none) 



Intercept 



-.107 
(2.229) 
C-.117J 

0.078 , 
(1.588) 
CO. 0821 

0.138 
(1.859) 
[0.097] 

0.808 ^« 
(2.791) 



0.106 
(2.248) 
[0.12O] 

-.025 
(0.531) 
[-.028] 

0.142 
(1.933) 
[0.102] 

0.642 
(2.247) 



mm 



mm 



0.166 
(1.941) 
(0.103) 

-.184 
(2.129) 
(-.113) 

-.015 
(0.112) 
(-.006) 

0.428 
(0.832) 



mm 



R* 

N-360 



.049 



.028 



.026 



p (one-tailed) < .10 
p (one- tailed) < .05 
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Table 8. Itorth Carolina Court Counselor's Intensive Proteotive Supervision 
Bxperinent: Regression Analyses of Three OuteoMe Variables for 
Four Original Sites Only for Period of Supervision and Op to One 
Year After for Closed Cases: Metric Regression Coefficients, 
t ratios (in parentheses), and Standardized Regresaion Coefficients 

fin bracketsl" 



Outcoae Variables 



Independent 
Variables 



DELOFP 



STATOFF 



Oroup 

(exper . •! » control«0) 



-.033 
(0.611) 
[-.036] 



-.061 
(1.076) 
[-.066) 



Age (years) at last 
birthday at entry 
into protective 
supervision 



-.039 , 
(1.511) 
(-.091] 



-.023 
(0.869) 
[-.054] 



Sex 

( f enale"! , *ale-0 ) 



-.240 
(4.109) 
(-.249) 



(1.848) 
[0.115] 



Race 

(blaclc*! .nonblack"0) 



0.062 
(1.075) 
[0.064] 



-.065 
(1.085) 
[-.067] 



Prior Delinquent 

Offenses 
( l"pr iors . O»none ) 



0.086 
(0.984) 
[0.059] 



0.141 , 
(1.590) 
[0.097] 



Intercept 



1.009 
(2.773) 



4t« 



0.593 , 
(1.562) 



R 

N»26S 



.093 



.028 



mm P (one-tailed) < .10 
p (one- tailed) < .05 
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One difference in Tables 7 and 8 relative to regression results in the 
March 1991 Phase in Interim Evaluation Report is that race of client reaches 
marginal statistical significance — indicating a slight increase in 
probability of recidivism for black youth clients for the supervision 
period but not for the cumulative period data. This is consistent with our 
earlier reports^ but the race variable was not even marginally significant 
for either period in the March 1991 report. 

Also consistent with earlier reports ^ the experimental treatment variable^ 
while having the expected effect in reducing delinquent referrals^ does not 
reaoh conventional levels of statistical signiticance in either Table 7 or 
Table 8. 

In our June 1990 Phase XI Final Evaluation Report, only one explanatory 
variable, client's gender, attained marginal statistical significance in the 
resireasion equations for the STATOFF outcome variable • This variable has 
increased statistical significance in Tables 7 and 8, In brief, being female 
increases the probability of a client being reported for status offenses 
either while under protective supervision or during the cumulative period. 

In addition, age and prior delinquent offenses now ate statistically 
significant in the STATOFF outcome variable for supervision-period results in 
Table 7 — the former in a negative and the latter in a positive direct ion^ 
respectively. By comparison, in the cumulative STATOFF equation of Table 8, 
the prior delinquent offenses variable is of only marginal statistical 
significance in the positive direction and age is not statistically 
significant. In its place, race of youth client has a marginally significant 
relationship to the status offense rate. 

The regression equation for the SUCCESS outcome variable in Table 7 shows 
results similar to those of the June 1990 report, with sex and race of client 
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strongly predictive of outcome judgments (female^ and nonblacka more likely to 
be judged aucceasCul) . However, being in the e2q>erimental group ia not 
significantly more likely to produce a judgment of succesj by the counselora^ 
net of other regresaor variables, in Table 7« This is consistent with the 
niean^*^fference comparisons reported in Panels A and B o£ Table 3 above » 

Slta*8pecl£lc AnsiXyaas, To ascertain the extent to which the foregoing 
results may be due to heterogeneity among the experimental sites, Tables 9 and 
10 report parallel regression equations to those, respectively, in Tables 7 
and 8 but with dummy-variable controls for the experimental locations. In 
these equations, Raleigh (District 10} is taken as the baseline or control 
location, and the effects on the outcome variables of the other sites are 
measured as deviatioas from Raleigh. Also, because only small numbers of 
cases have been closed in the four new experimental sites, only closed cases 
from District 18 (Greensboro), District 19 (Concord/Salisbury), and District 
28 (Asheville) are included in the regressions. 

It can be seen that the experimental site control variables drive the 
effect of the prior delinquencies and race variables to statistics 1 
insignificance in the supervision-period DELOFF equation in Table 9 as 
compared to Table 7. That is^ after introducing controls for the aites# the 
effect of the prior delinquent offenses and race variables on the DELOFF 
outcome become statistically insignificant. As in Tables 7 and 8, however, 
the gender and age regressors retain their signficance in the DELOFF equations 
of Tables 9 and 10. But none of the site-specific control variables are 
statistically significant in Table lO^nd only that for District 28 
(Asheville) is marginally siqnif leant in Table 9. This indicates that the 
regression relationships to this outcome variable differs very little by 
experimental site* 
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Table 9. Rorth Carolina Court Counselor's Intensive Protective Buo^rritiQu 
Bxperinent: Regression Analyses of Tliree OatooM Variables for 
Period of Supervision for Closed Cases Controlling for Oistriot: 
Netrie Regression Coefficients, t ratios (in pareatbeses) , and 

Standardized Regression Coefficients fin brackets!. 



Qutcone Variables 

Independent 

Variables DELOFF STATOFF SUCCESS 



District 18 

(Oist. 18»l,others"0) 



District 19 

(Dlst . 19«1 . others»0 ) 



District as 

(Oist . 28*1 , o thers>0 ) 



Group 

(exper . «1 » control«0 ) 



Age (years) at last 
birthday at entry 
into protective 
supervision 

Sex 

(feaale»l .aale^O) 



Race 

(black'l , nonblack-O) 



Prior Delinquent 

Offenses 
( 1 «pr 1 ors . O«none ) 

lnterce(.t 



0.045 

0.649) 

0.048] 

-.088 

1.124) 

-.081] 

-.101 , 

1.313) 

-.098] 

-.019 

0.359) 

-.022] 

-.048 
1.931) 
-.118] 



-.138 
2.381) 
-.148] 



0.064 

1.139) 

0.071] 

0.104 

1.217) 

0.076] 

1.017 
(2.900) 



0.046 

0.650) 

0.048] 

0.198 
2.502) 
0.181] 

0.238 
3.059) 
0.230] 

-.066 

1.247) 

-.075] 

-.008 

0.327) 

-.020] 



0.115 
1.969) 
0.121] 

0.017 

0.294) 

0.018] 

0.151 
1.757) 
0.110] 

0.195 
(0.549) 



-.077 

0.566) 

-.043] 

-.034 

0.225) 

-.017] 

0.068 

0.455) 

0.035] 

0.070 

0.695) 

0.043] 

0.025 

0.526) 

0.033] 



0.268 
2.380) 
0.149] 



-.162 ^ 

1.474) 

-.094] 

0.083 

0.504) 

0.032] 

0.132 
(0.194) 



N»265 



.083 



.080 



.043 



P (one-tailed) < .10 
p (one-tailed) < .05 



Table 10. Rorth Carolina Court Cooaselor's Xateoslve Proteotlvo Supervlalon 
BxperlMBt: Regression Analyses of Three Oatooae Variables for 
Period of Si^rvlslon and Up to One Year After for Closed Cases 
Controlling for District: Netrlo Regression Coefficients* t ratios 
(In parentbeses) . and Standardized Regression Coefficients 



Qutcoae Variables 



Independent 
Verifies 



DELOFP 



STATOPF 



District 18 

(Dlat . 18>1 , others>0) 



District 19 

(Dlst. 19"1 .otbers-0) 



District 28 

( Dlst . 28*1 , others-O ) 



Group 

<exper . -1 , control "0 ) 



Age (years) at last 
birthday at entry 
Into protective 
supervision 

Sex 

( f eMale-l , aaleoK) ) 



Race 

(black"l»nonblack«0) 



Prior Delinquent 

Offenses 
( laprlors , O°none ) 



0.036 

0.487) 

0.036] 

>.061 

0.740) 

-.053] 

-.087 

1.080) 

-.080] 

-.030 

0.S50) 

-.033] 

-.030 , 

1.514) 

-.092] 



-.'548 

4.081) 

-.247] 

0.037 

0.628) 

0.039] 

0.099 

1.105) 

0.068] 



«« 



0.053 

0.722) 

0.053] 

0.283 
3.427) 
0.245] 

0.288 
3.524) 
0.262] 

-.077 , 

1.384) 

-.083] 

-.022 

0.843) 

-.051] 



0.115 

1.876) 

0.114] 

-.001 

0.018) 

-.001] 

0.087 

0.959) 

0.059] 



Intercept 



R- 

N-265 



1.039 , 
(2.824) 

.103 



0.431 , 
(1.160) 

.099 



«» P (one- tailed) < .10 
p (one- tailed) < .05 
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In the STATOFF equations of Tables 9 and 10, the dummy-variable 
controls for Concord/Salisbury and Asheville are statistically significant and 
positive (thus indicating that these sites have higher status offense 
recidivism rates than the control site — Raleigh). These site control 
variables also drive the Table 7 effect of age, but not those of prior 
delinqpient offenses and sex, to insignificance. For Table 10, the site 
control variables drive the prior delinquencies effecc of Table 6 to 
statistical insignificance bat not that of sex. In addition, the group 
control variable in the STATOFF equation of Table 10 is negative and 
marginally significant, chus indicating that youths in the experimental group 
are less likely, net of the other variables, to recidivate with other status 
offenses in the cumulative period. 

Finally, in the supervision-period SUCCESS equation of Tables 9, 
introduction of the 3ite--pecif ic controls reduces the effect of the race 
variable, but not that of sex, to marginal statistical significance. Howeverr 
none of the site-'specif Ic controls or other regressors achieve statistical 
significance in the equation. In other words, net of the variables in the 
SUCCESS equation of Table 9, sex and race make a difference in counselor's 
judgments of whether or not the client successfully con^leted the supervision 
period — with females mo.e likely and blacks less likely to be judged 
successful « This may indicate counselor perceptions of greater <less) 
compliance and cooperativeness with their efforts on the part of females 
(blacks) • 

In brief, the most notable findings of adding the experimental site 
controls is that youth clients in the Asheville district are at marginally 
greater risk of delinquent offense recidivism, that youths in the 
Concord/Salisbury and Asheville districts are at greater risk of status 
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offense recidivism^ and that the experimental group treatinent effect becoiras 
foarginally atatiaticaXly significant in reducing status offense recidivism in 
the presence of site controls for the cumulative period. 

Predicting Successful Outcoiras in Intensive Supervision 

Based on previously reported field research conducted in our Phase II 
Evaluation Project (by Jay R. Williams)^ a number of characteristics of youth 
clients and/or their families that Intensive Protective Supervision Counselors 
believe to be predictive of t ^^eater or lesser probability of successful 
outcomes of Intensive Protective Supervision (IPS) have been identified in 
previous Interim Evaluation Reports* T.^ese beliefs, however^ have been based 
on informal observations and intuition on the part of the counselors. The 
question remains^ as to whether or not these characteristics actually could be 
used to identify which youth clients would benefit from assignment to the IPS 
program and which would not * 

To evaluate whether or not these counselor judgments could be articulated 
and shown to be predictive of outcomes of Intensive Protective Supervision, 
we previously constructed a questionnairer called the "Profile of IPS Clients 
and Their Families^" and requested that the Intensive Counselors fill out the 
questionnaire for each case in their program that has been closed to date. To 
fix ideas^ we asked each Intensive Counselor to think back to when a client 
became their case and to answer the questions in reference to what opinions 
they had about the client and their family within the first month of the 
assignment* With completed questionnaires now available for all previously 
closed cases, we continue to request that Intensive Counselors fill out the 
questionnaire for each case under their supervision — except that we now 
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jcequest that this be done approximately one month to six weeks after they are 
aaaigned the case. 

h total of 195 coirpleted Profiles for closed Intensive Protective 
Supervision cases now have been returned by the Counselors — including 165 
from cases at the four original experimental sites. This compares with 13S 
conq;>leted Profiles available for our October 1990 Phase III Interim Evaluation 
Report. Each Profile consists of 47 items or continua of characteristics^ 
with 5 equally-spaced boxes supplied for the counselors to rate the items for 
each youth client from one extreme to the other. For exan^le# the first item 
is: 



Runaway 
behavior is 
highly chronic 



Runaway 
behavior is 
nonexistent 



Another item is: 

Client is. 
not hostile 
at all 

A complete listing of the items was included in an appended copy of the 
Profile as part of our June 1990 Phase II Final Evaluation Report. 

Initially, these items were coded as rating scales with scores going from 
1 to 5- Implicitly, this assumes that the counselors divide eacM "continuum 
of characteristics'* in the Profile into equal intervals among the five boxes 
supplied for their ratings — in other words, that the scales are ••interval 
scales this assumption has been found to be justified methodologically 
provided that the items can be grouped into unidimensional scales. 
Accordingly, a very important step in the analysis of the items of the Profile 
is an assessment of dimensionality. 



Client is 
very hostile 
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For this purpose, we initially factor-analyzed all 47 items with a 
standard orthogonal principal components with varimax rotation algorithm. 
This yielded 11 significant factor dimensions (dimensions with eigenvalues of 
l.O or greater) accounting for 74.3 percent of the variation among all 47 
items. Based on previous analyses of these factors reported in our June 1990 
Phase II Final Evaluation Report, we expected that 4 of these 11 factors would 
be sufficiently unidimensional for further analysis. This expectation was 
corroborated. 

Accordingly^ we collapsed the original 11 factors into 4 unidimensional 
scales (containing 29 of the original 47 items) , which we labeled, 
respectively, parents^ defieieneies (comprising 12 items) , juvenilQ^g PCOfale»M 
(comprising 9 items) , juvenile /p arents^ 9gh99Ulig PfgfrJ.gm8 (comprising 16 
items), and previous services (comprising 4 items) — where, as noted in the 
Profile, the term "parent (s)" refers to the adult or adults who have 
responsibility for the youth clients. The first three of these scales contain 
one or more items than did the corresponding scales in previous reports. 
Hence, Table 11 contains a complete listing of the items belonging to each of 
the scales . 

Of these four scales, the last (orevious aSIisjjCfia) contains 4 
questionnaire items dealing with agency services previously provided to the 
client and her/his family. Thu parents' deficiencies sSSlfe continues to 
contain items having to do with parental resources, cooperation with the 
intensive supervision program, and extent of care for the client. This scale 
also contains the items concerning parents' substance abuse (including 
alcohol) and parents' involvement in criminal behavior. The lMVgnj,JTg^^ 
problems scale comprises items pertaining to the chronicity of the juvenile's 
runaway and ungovernable behaviors, his/her being at risk of delinquency. 



ERIC 



34 



Tabl9 11. Horth Cttrolina Court CooMelor's latmisive ProtseUvt $iipervi«loii 

BxperlMnt: Factor-Based Sealea and Thoir Coapone&t XtOM frea 
tha "Profllea of IPS Cllenta and Tlwir Pa^lllea* Inatmaant. 



Scale Coaponent Iteaa 

Parent's Deficiencies Parents are Involved in criainal behavior 

Parents will not participate at all in 

current services offered 
Parents are substance abusers (other than 

alcohol ) 
Parents are alcoholic 
Faaily has severe financial probleas 
Parents are very iaaature 
Parents are alcohol abusers 

Parents are umrilllns to cooperate with counselor 
Parents do not care about the client at all 
Parents are very apathetic 
Client has no place to call hoae (aoves a lot) 
Client has no significant adult to provide support 

Juvenile/Parent Client is at high risk to drop out of school 

Schooling Probleas Truant behavior is hi^ly chronic 

Parents have no poirer in the faaily 

Parents are "mm out" — no eaotional energy to 

help their child 
Parents provide no discipline 
Parents do not have any skills to carry thru 

with services offered 
Parents have no control over their children 
Faaily is extreaely "dysfunctional" 
Client is very dysfunctional at school 
Parents are enablers of client's truancy 
Parents are totally "helpless" 
Parents do not have any resources to carry thru 

with services offered 
Parents have no parenting skills 
Faaily has severe coaaunications probleas 
Client is very dysfunctional in the hone 
Client will not cooperate at all 



Juvenile's Probleas 



Runaway behavior is hi^ly chronic 
Ungovernable behavior is highly chronic 
Client's peer group Is into highly delinquent 
behavior 

Client's peer group is into highly chronic 

undisciplined behavior 
Client is at high risk for delinquent behavior 
Client is an alcohol abuser 

Client is a substance abuser (other than alcohol) 
Client runs away to things (e.g.. friends, 
adventure) as opposed to running away froa 
things (e.g.. abuse, sexual exploitation) 
Client is alcoholic 



Previous Services 




Various services provided client previously 
Services previously provided client were not at 

all successful 
Various services provided client's fully 

previously 

Services previously provided client's faaily were 
not at all auccessful 
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having delinquent peer3r being an alcohol or other substance abuser r and 
running "to thing3'' (as opposed to -away from** things) . Finally, the 
iuvenile/parents^ achoolina pspblems SUSSlS. contains items on the ohronicity of 
the truancy behavior of the juveniler her/his risk of becoming a dropout^ the 
parents' lack of parenting skills, power, and control, the dysfunctionality of 
the family, the juvenile's dysfunctionality in school, the parent's enabling 
of the juvenile's truancy, and the parents' being helpless and not providing 
discipline for the juvenile. Note that the last two scales divide the items 
nicely along the truancy versus runaway/ungovernable categories of status 
offenders • 

After constructing the foregoing scales, we sought to assess their 
predictive ability with respect to the three outcome variables for the 166 
closed cases in the experimental (IPS) group from the four original 
experimental sites £2£ Ul£ supervision-period qqA X£S£. £oXXa3SzS&- Xn 

other words, we sought to assess the ability of the factor-based scales to 
increase our ability to predict failure or successful completion of intensive 
supervision. Results for these predictive analyses are reported in Table 12, 
which displays the estimates of regression models in which the foregoing 
factor-based scales are entered together with the demographic and court- 
related explanatory variables of the regressions previously reported in Table 
8 (and Table 7 for the SUCCESS outcome variable; note that there is no dummy 
variable for experimental versus control group in the regressions of Table 12, 
as all cases are from the experimental group) • 

Relative to the demographic and court-related variables, it can be seen 
that both the client's sex and prior delinquent offenses are statistically 
significant in the DELOFF equation of Table 12 • That is, females are less 
likely to be referred for delinquent offenses, while those with prior 
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Table 12. North Carolina Court Connsalor's IntsMive Protoetive Si^arvltioD 
BxperlBoats itogrossloa Analyses of Tbree Outeoae VSrlaUes for 
Closed Bitperlaental Group Cases Using Factor Soore Variables froa 
Connaelor Profiles for Period of Sttpervislon and Op to One Year 
After Netrio Regression Coefficients, t ratios (in parentbeses) , 

api^ S^ffftfU^d^gyd ^^ess^off ^ff ji94eft^f [^ft brftcy^f^^] 



Outcoae Variables 

Independent 

Variables OBLOFF STATOFF SUCCBSS 



Parents ' 
Deficiencies 



Juvenile's 
Probleas 



Juv/Parent 
School ins 
Probleas 

Previous 
Services 



Age (years) at last 
birthday at entry 
into protective 
supervision 

Sex 

( f eaale- 1 . Male<*0 ) 



Race 

(black«l ,nonblack«0) 



Prior Delinquent 

Offenses 
( loprlors , O«none ) 

Intercept 



R 

N-ld6 



-.000 

0.169) 

-.016] 

0.019 

4.919) 

0.366] 

-.001 

0.365) 

-.036] 

0.003 

0.549) 

0.040] 

-.015 

0.585) 

-.042] 



-.142 

2.052) 

-.151] 

0.027 

0.384) 

0.027] 

0.236 

2.489) 

0.181] 



0.092 
(0.245) 

.252 



«« 



«* 



0.001 

0.327) 

0.032] 

0.017 
4.160) 
0.320] 

0.006 

1.728) 

0.174] 

-.004 

0.888) 

-.067) 



-.053 

1.995) 

-.149] 



0.131 
1.803) 
0.137] 

-.068 

0.940) 

-.068] 

0.097 

0.976) 

0.073] 



0.137 
(0.350) 

.203 



mm 



0.006 

1.145) 

0.116] 

-.008 

1.806) 

-.146] 



mm 



-.014 

3.219) 

-.340] 

-.000 

0.025) 

-.002] 

-.002 

0.049) 

-.004] 



0.036 

0.420) 

0.034] 

0.025 

0.285) 

0.022] 

O.0S6 

0.474) 

0.037] 



1.457 
(3.151) 

.123 



p (one- tailed) < .10 
p (one-tailed) < .05 
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delinquent offenses prior to protective supervision are more likely to be so 
referred. In addition, we find a powerful effect of the juvenile's problems 
scale in the DELOFF equation of Table 12 -•^ suggesting that high values on 
this factor^based scale are predictive of referrals for delinquent o£fen<^es 
during the supervision period. The effect of this scale also drives the 
marginally statistically significant effects of age and race in Table 8 to 
insignificance in Table 12. 

In the STATOFP equation of Table 12, coefficients of the age and sex of 
youth client variables are statistically significant* In this equation, 
however, the marginally significant effect of prior delinquent offenses noted 
in Table 8 appears to be mediated by two of the factor-based scales — 
juvenile's problems and iuvenile/parent schooling problems. 

By comparison, neither of the sex and race variables in the SUCCESS 
equation — which were statistically significant in Table 7 ~ are found to be 
significant in the corresponding equations of Table 12. In other words, the 
explanatory effects of these variables in predicting outcomes of the 
experimental treatment are entirely mediated by two factor-based scales ~ 
juvenile's problems and juvonile/parent schooling problems. 

Overall, it must be concluded that the factor-abased scales add 
considerable explanatory power to the demographic and prior court history 
variables. For all three outcome variables^ the coefficients of determination 
adjusted for degrees of freedom <R^3) are increased by several magnitudes in 
Table 12 as compared to Tables 7 and 8. Specif ically^ the percentage of 
variance explained for the DELOFF variable in Table 12 is 2.7 times that in 
Table 8^ the percentage for the iTATOFF variable increases 7.25 times that in 
Table Bf and the percentage for the SUCCESS variable goes up about 4.7 times 
that in Table 7. It must be concluded that these results demonstrate that 
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9ub3tantlal io^rovements in predictive accurac for caae outcomes in intensive 
supervision can be obtained by taking into account juvenile clients^ scores on 
the factor^based scales we have constructed. 

Changes in Treatment Effectiveness Cross-Tabulatioa Analyses 

In our March 1991 Phase III Interim Evaluation Report^ we noted new 
evidence of a tendency towards deterioration of the experimental treatn^nt. 
Specifically^ the difference in the effectiveness of IPS relative to RPS seems 
to have been greater for youth clients who were under protective supervision 
during the first year-and-a-half of the experiment (roughly November 1987 to 
May 1989) than those who were under supervision during the second year--and«*a- 
half (roughly June 1989 through December 1990) . In other words^ mean 
differences in the outcome variables were larger for the "early'' project 
participants than for the **late** participants. 

imother mode of analysis of these changes is displayed in Table 13 which 
contains cross-tabulations of the OELOFF outcome variable (supervision-period 
only) for the first 50 closed cases in each of the experimental and control 
groups in the top panel and the last 50 closed cases in each group in the 
bottom panel (to maximize comparability, only cases from the four original 
experimental sites are included in the table) * As all of the 100 cases in the 
top panel were under protective supervision during the "early" phase of the 
project (the first year-and-a-half ) and all 100 cases in the bottom panel 
similarly were under supervision during the "late" phase «the last two years)/ 
these cross-tabulations are appiopriate for assessing er rly vs. late 
effectiveness of IPS relative to RPS. 

In the top panel of Table 13, it can De seen that/ relative to the goal of 
reducing delinquent court retetrais ot youths under protective supervision/ 
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t9bl9 13. Worth Carolina Court Cooiwolor*s Inteoalvo Protsotivt a ^*v4«tiui 
BxporlMBt: Croaotabalatlono of DBKOPF Outoiae variable v.. 
BsporlMmtal vs Control Group for Four Original Sltoa Only: 
Barly (Flrat Year and a Balf ) and Late (Second Year and a Balf ) 

Early Early 
Bxperlnental Control 



No Delinquent 44 34 

Offenaes (88. 0«) (68.<») 

One or More 6 16 

Delinquent Offenses (12.0«) (32.0%) 

Total 50 50 



Late Late 
E»perlnental Control 



No Delinqent 38 36 

Offenses (76. 0«) (72. OX) 

One or Itore 12 14 

Delinquent Offenses (24.0K) (28.0«) 

Total 50 50 
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the experimental (IPS) treatment was quite effective during the early period 
p£ the project: the ratio of youths who completed supervision with no 
delinquent referrals to those with one or more referrals was 7,33 (88% to 12%) 
in the early experimental group as compared to 2,13 (6S% to 12%) in the 
early control group. By contrast ^ the corresponding ratio in the late 
experirontal group was 3.17 (76% to 24%) and that in the late control group 
was 2*57 (72% to 28%). As was noted earlier in the difference-of -means 
an€.lyses, the cross-tabulations in Table 13 exhibit both an increase in the 
effectiveness of the control (RPS) treatment and a decline in the 
effectiveness of the experimental (IPS) treatment. 

Cross -tabulations for the other two outcome variables (STATOPF and 
SUCCESS) Show similar patterns of change in the project outcomes for early 
versus late comparisons. 

We already have observed that some improvement in the effectiveness of the 
RPS treatment could be associated with RPS counselors learning and adopting 
••experimental** procedures. The decline in effectiveness of the IPS treatment 
is more puzzling. 

To examine the possibility that some of this decline is due to changes in 
IPS personnel that have occurred at some of the sites, we confuted cross- 
tabulations like those in Table 13 for a site that has had experienced and 
stable staffing of the IPS counselor position throughout the project — 
District 10 (Raleigh) , Again, effectiveness declined for the late 
experimental period as compared to the early period. A further cross- 
tabulation of outcome variables for all closed cases from two districts with 
stable staffing throughout the three-year project period {District 10 - 
Raleigh and District 28 - Asheville) vs. two that experienced some personnel 
changes <District 18 - Greensboro and District 19A - Concord/Salisbury) 

ERIC 



9U9903t3 that stability in counseling staff might account for a sioaXl part o£ 
the change in outcome effectiveness but not much. Finallyr although the 
numbers of closed cases from the four new sites are not yet sufficiently large 
to draw strong conclusions, it appears that their outcome effectiveness is 
more like that of the "late" period of the four original sites than the 
"early" period of these sites • 

Of the various hypotheses (to explain the decline in effectiveness of the 
intensive treatment procedure) stated earlier herein^ the only remaining one 
is that there was some decline in enthusiasm on the part of the intensive 
counselors as the novelty of the ipS process declined and the program became a 
routinized part of the juvenile court bureaucracy. Based on the numerical 
analyses we have conducted and a description of the time secjuence of project 
events and training seminars provided by Administrative Office of the Courts 
personnel, it is plausible that the development of the IPSP project and its 
procedures during the first year-and-a-half of the experiment — with all of 
the attention from Administrative Office of the Courts personnel and the 
initial evaluation research project personnel — led to a higher level of 
enthusiasm and effectiveness among intensive counselors than that sustained 
more recently- This hypothesis is consistent with the apparent lower 
effectiveness of IPS in the four new sites — where IPS counselors similarly 
have received special training in therapeutic procedur;L-3 but have not 
benefited from the excitement of the "start -up" phases of the project. 

In the realm of experiments involving human subjects^ the tendency of 
outcome variables to react to anv change in attention given to subjects is 
known as the ••Hawthorne Effect." This is due to the fact that experimental 
subjects may take the fact that they have been singled out for extra attention 
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i^s indicative of interesn in their personal welfare and reapond with 
behavioral changes. 

Conventional redee|rch designs atten^t to control for the Hawthorne Effect 
on experimental subjects by use of a ''control groups** as in the present 
project* However, as Rossi and Freeman (1989, p. 248) note, in evaluation 
research experiments ^v^py aspect of the intervention delivery system can 
affect the outcome of the intervention* In particular, based on the cross* 
tabulation results noted above, it is plausible that, for IPS to achieve its 
maximum effectiveness, intensive counselors must receive continuing and 
periodic ••shots'* of training and the attention associated therewith. That is, 
it may be the case that the Hawthorne Effect has affected the '•early'* vs* 
"late" outcomes of the IPS project — not through the experimental subjects 
(the youth clients), but r,^ther at another level — that of the -ntensive 
counselors « (This hypothesis of a Hawthorne Effect for intensive counselors 
also is explored in the draft chapter attached as Appendix A.) 

At the level of the intensive counselors themselves, the flip side of the 
Hawthorne Effect is "counselor burnout" or decline in "extra efforts to 
••save" the juvenile client from further status offending and/or delinquent 
behavior* While we have little evidence with which to test this possibility 
(as we were not commissioned to study the counselors), it is plausible that 
the effects of counselor burnout would be reflected in the assigning of 
services to the juvenile clients. Following this logic, it is significant 
that, for the first 100 cases (i-e., the early cases) in Table 13, intensive 
counselors assigned three or more professional diagnostic and/or therapeutic 
services to 62% of their cases compared to the regular counselors assigning 
three or more services to 24^ of their cases. By comparison, for the last 100 
cases <i.e., the late case) in Table 13, intensive counselors assigned this 
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level of services to 42% of cases compared to 28% for the regular counselors « 
Again^ there Is a slight increase in the numbers of services assigned by 
regular counselors as they presumedly atten^ted to adopt **intensive 
procedures'* during the latter part of the experiment* But^ even more 
dramaticallyr there is a large drop in the frequency of assignment of services 
by the intensive counselors. Thus^ on the assun^tions that these behavioral 
changes in numbers of services assigned to the juvenile clients or their 
families reflect changes in intensive counselor efforts to change the behavior 
of the clients and that the latter reflects levels of counselor enthusiasm and 
morale^ ve surmise that there was indeed at least some substantial level of 
counselor burnout in the latter period of the IPSP experiment as compared to 
the early period ♦ 

Conclusion 

With a steadily increasing numb'^r of closed cases ^ some of the findings of 
statistically and substantively significant differences between the 
experimental and control group clients as reported in our previous reports 
continue to held up^ whereas others do not. 

perhaps most important is that we continue to find a clinically 
significant mean difference treatment (i.e., intensive supervision) effect on 
the important delinquent offense ouccome variable for those youths with no 
prior delinc[uent referrals. Currently/ our estimate for this group of youths 
is that membership in the Intensive Protective Supervision treatment group 
results in a reduction ai about i? percent in the occurrence of referrals to 
the court for delinquent offenses (jiurjinq supervision period — relative to 
the rate observed in the control (Regular Protective Supers sion) group. EfljL 
ih& c umulative period {supervision period plus up to a one-year follow-up) r 
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this reduction is SR Itlfi order Qt 21 percent , Eaqperimental treatisent effects 
on this Qrder of mangnitude in juvenile and criminal justice research 
generally are considered of both policy and clinical (i.e., case treatment) 
significance. 

SaSSlDSi9 youths with prior delinquent court referrals are likely to 
recidivate with new delinquent referrals fiSxJLY (i*e.r during the protective 
supervision period) regardless of group membership {experimental or control) . 

Third , experimental (IPS) treatment appears to have a tSQISL laatinq effect 
in reducing delinquent and status offense referrals than control (RPS) 
procedures . 

Fourth s the primary cumulative impacts of experimental relative to control 
procedures are 3,Qwer nonfelonv delinquency. tIFWanCYi SOA yWT)«^ftY gftCidiTtgrn 

Fifth , with respect to counselor judgments of whether the protective 
supervision assignment was successful overall for a client, we continue to 
find that the Intensive Protective Supervision Counselors tend to rate a 
higher percentage of their clients as successful than do "he Regular 
Protective Supervision Counselors. At the same time^ however^ it should be 
noted that this tendency has declined by more than 75 percent as con^ared to 
that observed in the Phase I evaluation — i-»3rhap3 indicative of more 
re4:ilisti- assessments of the clients by the Intensive Counselors — and also 
becomes statistically insignificant when other youth characteristics are 
controlled* 

Sixth s in our efforts to estimate optimal predictive regression equations 
for the outcome variables^ we find the following variables are predictive of 
youth clients being referred to the court for delinquent offenses either 
during supervision or in the cumulative period: being male and younger at the 
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time of entry into protective supervision. Being black and having been 
referred to the courts for prior delinquent offenses also is predictive of 
delinquent offenses during the supervision, but noi: the cumulative/ periods. 

By coiq^arison^ being female is the most consistently significant 
predictor of status offense recidivism during or after the supervision 
period. Other factors that affect this outcome variable are: being younger, 
having prior delinquent referrals, and being resident in the Asheville or 
Concord sites, interestingly, after controlling for experimental site, being 
a member of the experimental group has an impact in reducing the status 
offense recidivism rate* 

For counselors' judgments of clients' having successfully conpleted the 
supervision period, the most important predictor is being female — with being 
nonblack also of slight significance. 

^QVf>n^h, our efforts to develop scales based on counselor assessments that 
can be used to in^rove predictions of outcomes of intensive supervision 
continue to be promising. Multi-item scales comprising counselor assessments 
of parents' skill and resource deficiencies, juvenile's personality, 
interactional, and behavioral problems, and juvenile/parents' schooling 
problems are successful in substantially xncrea::ing our ability to predict 
outcome variables for those youths assigned to Intensive Protective 
Supervision. 

Eighth > comparisons of "early** vs. ••late** outcomes for the experimental 
and control groups suggest that, in order for IPS to achieve it maximum 
effectiveness relative to RPS^ it may be necessary to provide IPS counselors 
with periodic training (in counseiinq and therapeutic procedures), attention, 
and morale boosting. The data are consistent with a hypothesis of a Hawthorne 
Effect on the enthusiasm of the intensive counselors during t>*e early phase of 
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the project — which^ in turn, led to a greater freq[uency of asaignment of 
larger niunbers of services to the juvenile clients and an apparent 
corresponding reduction in progression to delinquent referrals for these 
youths. Equivalently, during the later phase of the experintent^ there appears 
to have been a burnout on the part of the intensive counselorsr leading to the 
less frequent assignment of services <and presumedly^ less heroic efforts on 
the counselors' part) and a corresponding reduction in effectiveness of the 
intensive supervision procedures* 
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